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ANNUAL EVALUATION REPORT OF THE HAWAII CURRICULUM CENTER 
ENGLTSH PROJECT FOR J 968- 69 



Foreword 



This annual report is intended to provide project staff, participating 
teachers and administrators, funding agents, and interested educators and 
laymen with a descriptive overview and preliminary evaluation of the English 
Project to date. There are seven sections in the report: 1) Suimnary, 

Conclusions, and Recommendations, 2) Overview of the English Project, 3) Lan- 
guage Skills Subprogram, 4) Literature Subprogram, 5) Language Systems Sub- 
program, 6) Evaluation of the Evaluation, and 7) Appendices (available upon 
request) . 

Preparation of this report required the assistance of many people. Special 
thanks are due to Anne Nicol and Carol Smith for their evaluation assistance 
during the year, Edith Kleinjans and Edith Louis for their editorial work, 

Joy McLarty and David Chang for data analysis, and Pat Zakahi and Vivian Flake 
for typing this report. 
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I. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 



A. SUM^IARY 



This annual report contains data collected and analyzed by the English Project 
evaluation staff of the Hawaii Curriculum Center from September 1968 through 
August 1969, 

Curriculum planners and project administrators met with the evaluation 
specialist in determining those areas in most critical need of evaluation. An 
evaluation design was then prepared, reviewed, and used as the basis for the 
evaluation activities throughout the year. 

Attention was given to describing the background situation in which the project 
operated, and to evaluating the program's activities and preliminary outcomes, 

B, CONCLUSIONS 

Shown below is a set of tentative conclusions regarding the \^awaii English 
Project. Conclusion one is related to the entire English Project and the other 
nine are confined to the Language Skills Subprogram, 

1, Visiting scholars from the mainland and other countries who have observed 
the English Project have been favorably impressed. Among the things 
frequently praised were the design statements for the subprograms 
(Language Skills, Literature, and Language Systems), which they judged to 
be well formulated and soundly based in scholarship. The variety and 
innovativeness of many of the materials and instructional activities 
produced in all three subprograms also drew much favorable comment. 

2, Students, teachers, principals, parents, and visitors expressed generally 
enthusiastic attitudes toward the Language Skills subprogram. Students 
inter.iewed expressea a mre frequent preference fcr language arts 
activities than for any other subjects, and in almost all cases they felt 
they were succeeding in the programs. Most of the teachers and principals, 
while aware of the extra work required to individualize instruction, felt 
they were accomplishing more with the program than through traditional 
methods. The majority of parents interviewed thought the program was 
helping their children to learn, Children advanced noticeably in language 
skills, responsibility for their o>m learning, and interest in school. 
Visitors were generally impressed with the programs, particularly the 
individualization of instruction and the adequacy and variety of the 
materials. Two problems noted by visitors were the dislike of a few 
students for peer teaching and the amount of record-keeping done by some 
teachers, 

3, Children in the skills program have made substantial academic progress 
during the year. Sixty per cent of all participating kindergarten children 
this year had a reading knowledge of over 60 wordsj two kindergarten 
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children read over 35 books. Forty-six per cent of all participating 
first-grade children read over ten books; 11 percent read more than 35 
books- In writing, 50 per cent of all kindergarteners and 83 per cent 
of all first graders could copy four word sentences accurately from a model 
in cursive writing, 

4, A tremendous range was found in the kinds and numbers of skills programs 
needed, in numbers of days required to complete the programs, and in 
achievement by the end of the year, Materials were available for 
children at all levels, from a letter discrimination program successfully 
completed by all kindergarten children to a comprehensive reading kit 
designed to take children to a seventh grade level of rec.ding. 

5, Children in the progrs.m have also made substantial progress during the 
year in self-directed learning skills. The greatest improvement occurred 
in their ability to plan at least one language arts activity each day. 
Improvement was also noticeable in their ability to change activities 
without teacher direction, to keep trace of their place in programs, and 
to mark their own progress in their record folders. They shoved least 
gain in accepting tutoring from other children, 

6, Low correlations of students’ sex, IQ, and socioeconomic status with 
achievement in reading, writing, typing, and listening/ speaking have been 
found for the children in the Language Skills subprogram. This finding 
suggests that success in these programs is independent of certain 
conditions beyond the child’s control. For example, children from lower 
socioeconomic levels have about the same chance to succeed in the program 
as children from upper socioeconomic levels, 

7, The correlations of student progress in reading, writing, typewriting, and 
1 istening/ speaking are relatively low, with the exception of a moderate 
correlation betw3 n reading and listening/speaking. This means that it is 
not necessary that a child perform well in one area to be able to perform 
well in another; hn has multiple opportunities to succeed. This variety of 
possibilities is one measure of an individualized instructional program. 

8, Students responding to a media preference inventory indicated that the 
typewriter was favored most and flocked letter cards least. Nevertheless, 
each of the typos of media in use was liked by at least half of the children. 

9, The experiment of combining kindergarten and first-grade students in the 
same classroom was highly successful. Not only did kindergarteners double 
their achievement In certain areas of reading over last year but the first 
graders too made substantial progress. 

10. Peer teaching at the kindergarten and first-graae level proved to oe an 

effective method of learning. Post-test gain scores showed improvement not 
only for the learner but also for the tutors, supporting the notion that one 
learns by teaching others. More experiments are needed to determine what 
factors contribute to successful and unsuccessful peer teaching. 
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C. RECOMMENDATIONS 



The following reconunendations are based upon this year's evaluation. 

1. Planners need to develop a clearer set of objectives in the area of 
self-directed learner skills of students. 

2. Planners in the Literature and Language Syistems subprograms should 
plan their materials and instructional procedures to accommodate the 
self- directed learners being prepared in the Language Skills subprogram. 
There should be an articulation among the subprograms that enhances 

a child's growth in independence throughout his years in school. Like- 
wise it is recommended that leaders within the Department of Education 
consider the articulation of this individual iied instructional program 
with the total school curriculum, 

5. Plcnners in all three subprograms should re-examine tho value of each 
unit being developed to judge not only its scholarly soiaidness but 
also its relevance to the children of Kawaii and to their particular 
grade level. Evaluators also need to develop methodologies useful in 
answering these basic questions. 

4. A need exists for the section chiefs in all three subprograms of the 
English Project to determine more clearly what outcomes in students' 
behaviors are expected upon termination of their programs. For example, 
a clearer definition and elaboration of "sixth-grade level of profi- 
ciency in language skills" would be most useful in giving focus and 
direction to all sets of materials. 

5. New experiments in peer tutoring are needed to determine what specific 
combinations of variables contribute to its success or failure. 

6. Attention should be given to the development of a more systematic and 
timely feedback of information from evaluators to teachers, planners, 
and administrators. 
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II, OVERVIEW OF TliE ENGLISH PROJECT 



A, DESCRIPTION OF THE HAWAII CURRICULUM CENTE R AND TH E ENGLISH PROJECT 

The English Project is the major curriculum development effort of the Hawaii 
Curriculum Center, Drawing upon the resources of its sponsors, the 
Department of Education and the Universit/ of Hawaii, and assisted by a grant 
under Title III of the Elementary and Secondary Education Act, the Center 
brings together master teachers, academic scholars, and curriculum specialists 
to work on the problems of upgrading the school curriculum. 

The English Project, begun in 1966, was charged with producing a tested curric- 
ulum in English for all grades from kindergarten through high school, together 
with a plan for its installation in the schools of Hawaii and dissemination 
elsewhere,^ The project employs a staff of some fifty persons, including 
university specialists and master teachers in the fields of English, writers, 
evaluation specialists, media technicians, and clerical staff. The output of 
the project will be substantial*, materials for students; background materials 
for teachers; appropriate kinds of tests; supporting e<|;uipment and media; and 
teacher training programs, 

in defining the field of English for the schools, curriculum planners undertook 
to articulate a coherent theory which would logically incorporate and integrate 
those matteis that belong to English and elindnate those that do rot. They 
adopted a view of school English as inquiry into the English language and into 
literature as the creative usu of the language. Instruction in language skills 
is subsumed under the study of language and literature after children achieve 
basic literacy. 

The curriculum under development has three related strands. (See Appendix 1,) 

All beginners enter a Language Skills subprogram whose goal is effective commu- 
nication in language. Children select and enter the various skills sequences 
at their own level and move ahead at their own rate with the help of many 
learning tools -- Including card stacks, card readers, playback devices, 
typewriters, films, and games -- until they reach a stage of independent 
learning in English (defined as sixth-grade competence in reading, listening, 
speaking, and writing). Literature is introduced early via taped selections 
for listening. Then, as the children's need for work on skills lessens, their 
time spen^- with literature increases. 

The Literature subpre-gram has as its goal for students engagement with literature 
in a progressively discriminating way, until at length the/ turn to it for 
pleasure and instruction, have a mature and eclectic taste for it, and are able 
to respond to its infinite varieties pnd values. 



^By action of the 1969 Legislature, the scope of the project was reduced to 
grades K-6 only, Kork on the curriculum for grades 7-12 is suspended for fiscal 
year 1969-70. 
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The subprogram in literature is divided into six consecutive "bands” keyed 
loosely to grade levels with attention to interest and ability. Although 
the performance goals for each band focus on specific literary concepts, 
selections at lower levels are centered around content, at higher levels 
around literary concerns* 

The selections distributed throughout the curriculum represent world 
literature including legendary materials; Hawaiiana; great works in the 
British and American traditions; contemporary works written in English from 
all over the world; translations of works from Europe, the Orient, and the 
Pacific; poetry and a variety of non-fiction. Activities are selected to 
encourage engagement with literature, 

The Language Systems subprogram is a sequence of inquiries into language 
behavior which Incorporates the insights and processes of modem linguistic 
science and related studies. The elementary segment (beginning at grade four) 
examines the nature of communication in general through investigations into 
areas such as animal communication, sign languages, the language of advertising, 
an international language (Esperanto), non verbal communication, and secret 
codes. The junior high program will deal with the central proble:hi. and 
concerns of linguistics itself -- in particular, man’s capacity to construct 
and internalize at a very early age a highly intricate set of rules by which 
he is able to understand and generate an infinite variety of sentences. The 
proposed senior high school program will focus on language and culture. It 
will help the student to synthesize what he has learned earlier and to consider 
the overlap of linguistics with other disciplines -- psychology, anthropology, 
sociology, mathematics, and literature. 

The whole English curriculum is conceived as a continuum rather '"han a series 
of grade-bound segments. There will be a wide diversity of materials, 
selections, and activities, so that students, regardless of their ability, will 
find a pattern of study attractive to them individually. 

8, RELATION OF THE ENGLISH PROJECT TO OBJECTIVES OF TITLE 111 OF THE ELEMENTARY 
AND~^CONDARY~ EDUCATION ACT 



The English Project blends the purposes of Title III legislation and Hawaii 
Department of Education objectives with more particular objectives of its own. 
The thrust of Title III programs is to foster innovation in education; the 
Department of Education alms to provide quality education for all students ar.a 
to remedy inadequacies in school English programs. 

Title III objectives are set forth in A Manual for Project Applicants and 
Grantees . 

The Title III program ... is designed to encourage 
school districts to develop imaginative solutions to 
educational problems, to more effectively utilize 
research findings, and to create, design, and make 
intelligent use of supplementary centers and services 
.... Projects may be developed which (1) invent a 
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creative solution to a problem, (2) demonstrate an 
exemplary program which might be suitable for wide- 
spread use, (3) adapt an exemplary program to local 
requirements and organize its incorporation in the 
education program. Many projects will, of course, 
combine these functions. ^ 

The Hawaii Curriculum Center was established to assist the Department of 
Education in meeting Title IIPs triple challenge: to invent, demonstrate 

and implement innovative solutions to persistent problems in schooling. 

The Center is itself a unique attempt to create an environment conducive to 
fruitful collaboration among those in the Department of Education and in the 
University who have knowledge useful in curriculum building. 

1. Inventing Solutions 

The English Project from its inception has striven to meet the challenge 
of inventiveness in remedying the serious weaknesses in language arts 
programs cited in the Curriculum Survey Report of 1965. (See Appendix 2.) 
First the staff undertook to lay a comprehensive theoretical base upon 
which to build the design of the new curriculum. The design then accom- 
modates the theory to the notion of schooling as apprenticeship in 
learning, to knowledge of how children learn, and to the particular needs 
of Hawaii's children. The design incorporates fresh approaches throughout. 
Among the salient emphases incorporated in one or more strands of the new 
curriculum are the following: 

Individualization of instruction through a range of learning 
tools, organizational arrangements, and opportunities for 
self-choice, self-direction, and self-tracking 

Specificity of instructional objectives and achievement levels 

Built-in evaluation 

Inductive and discovery approach to learning 

Accivity-centercd learning through games, drama, writing, and 
other creative activities 

Greater use of non* text modes of educational presentation 
Peer teaching 

Effective early education to obviate the need for remedial 
education 

Other innovations are in the curriculum content itself, such as the unique 
collection of Pacific literature and the completely new content of the 
Language Systems units. 



^Washington, D.C.: U.S, Office of Education, 1967, p, 1. 
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The use of a systems approach to oper tions, incorporating such 
management techniques as PPBS (Program Planning and Budgeting System) 
and PERT (Program Evaluation and Review Technique), is also a new 
departure in curriculum building in Hawaii. 

2 . Demonstrating Solutions 

The exemplary English program being developed by the project staff was 
tested and demonstrated in sixteen classrooms in six schools designated 
as field schools. As more materials are readied, the scope of the demon- 
stration will expand until finally the curriculum is part of the regular 
instructional program in Hawaii’s public schools and those private ones 
that elect to use it. During the past year hundreds of people from 
Hawaii, other states, and foreign countries have visited the demonstration 
schoolij to see the Language Skills subpiogram in use by children. Their 
comments are summarized in Appendix 3. The English Project is also being 
widely publicized through a large-scale display of materials that has been 
shown at conventions, PTA meetings, and other events and places throughout 
the State. Slide showings and presentations have also been made in }iawaii 
and on the mainland. 

3. Implementing Solutions 

The nature of the English Project as a supplementary activity to ongoing 
operations of the Department of Education has value not only in assuring 
the preparation of a curriculum adapted to local requirements, but also in 
facilitating its installation in the schools. hTien the new curriculum is 
approved for widespread use, it will be implemented by the Department of 
Education with the help of the Curriculum Center according to a jointly- 
developed comprehensive plan for training teachers, supplying materials, 
and phasing the new curriculum into the schools. 

C. EVALUATION MODEL FOR THE ENGLISH PROJECT 

The evaluation nvodel (attached as Appendix 4) received detailed review by an 
evaluation committee and has undergone four revisions throughout the year as 
new insights have been gained by the evaluation specialists and project staff. 
This document is essentially a model for evaluating the English Project 
rather than a model of the evaluation"tTiat has actually been performed. It 
specifies some activTTies that the evaluation specialists believe should be 
performed as part of a comprehensive evaluation, even tnough some of these 
activities have not been undertaken this year. The intention is that next 
year’s evaluation will more closely approach the model while at the same time 
the model will be reshaped to acconunodate the realities of the environment. 

This report will take up the three subprograms that comprise the English 
Project curriculum. Although the model describes areas *o be evaluated during 
the complete cycle of development, the evaluation during 1968-69 focused 
primarily on the Language Skills subprogram, since it was already in operation 
in field schools. 
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III. LANGUAGE SKILLS SUBPROGRAM 



A. THE DESIGN OF THE EVALUATION 



The Language Skills evaluation design enunciates the premises which guide 
the evaluation, the purposes of the evaluation, the areas selected for 
evaluation, the instruments used and their purposes, the personnel partici- 
pating, and the procedures for recording and reporting practices. A detailed 
summary of the evaluation design is attached as Appendix 5, 

The primary purpose of the evaluation effort was to provide information 
useful in revising the curriculum programs and assessing progress to date. 

The evaluation focused upon the environment in which the curriculum was 
taught, the operational procedures, and the prototype outcomes. Instruments 
used included classroom records, classroom observers, feedback sheets, 
diagnostic tests, performance tests, academic potential measures, case studies, 
interviews, Q-sorts, questionnaires, teacher ratings, and equipment utilization 
counters. 

In addition to the Curriculum Center staff, two VISTA workers and eight parents 
hired as data collector.s assisted in the evaluation by observing field-school 
classrooms and recording pertinent data. 

Quarterly evaluation reports had been planned, but prompt reporting of 
information to the project manager and planners at weekly staff meetings 
proved more timely and helpful in guiding revisions of materials. 

The results of the Language Skills evaluation are reported here in three 
categories; (1) environmental variables (those factors affecting the context 
in which the program operates)^ (2) program variables (the goals, objectives, 
activities, and materials in the program and the participants' perceptions of 
and attitudes toward theiri) and (3) prototype outcome variables (the effects of 
the trial of prototype materials). 

B. ENVIRONMENTAL VARIABLES 

1. Field school communities 



The Language Skills materials were tried out in four field schools on 
Molokai (Kilohana, Kaunakakai, Kuaiapuu, and Maunaloa), at Kalihi-Uka school 
in Honolulu, and at the Laboratory School of the University of Hawaii during 
the school year 1968-69. These six schools encompass a spectrum of social, 
economic, educational, ethnic, and ability differentials, (See Appendix 6 
for details and statistics.) Molokai is a rural island where a nixed ethnic 
population makes a living largely from pineapple fields, cattle ranches, and 
the sea, Roughly half of the fathers of school children there have finished 
high school, and the average family income is under $6,000 a year. The 
Kalihi-Uka community presents a similar ethnic and economir mix in a densely 
populated urban environment. 
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The University Laboratory School children span the whole gamut of socio- 
economic strata with a large proportion from high occupational and 
educational levels. Half the fathers of kindergarteners and first graders 
are in professional, executive, and skilled work. Over 40 per cent have 
gone through college or beyond, and only 11 per cent did not finish high 
school . 

2. Students 



Kindergarteners and first graders participating in the trial of the 
Language Skills materials totaled 264 in the four Molokai schools and 201 
in Kalihi-Uka. Laboratory School tryouts involved 108 children, 
including 34 second graders in combined K«2 classes. The academic 
potential of these children as measured by the Kuhlmann-Anderson Test 
(7th V iition) averaged between 95 and 102 for kindergarten classes in 
Kalihi-Uka and the Molokai schools; 112 in the Laboratory School. First- 
grade averages ran from 96 to 109 in Kalihi-Uka and Molokai classes, the 
average was 125 in the Laboratory school. (See Appendix 7.) 

3. Teachers 



Nine teachers at Kalihi-Uka, 15 on Molokai, and four at the Laboratory 
School participated in the testing program. The pertinent variables 
selected for teachers were years of experience, education, participation 
in tiie 1967-68 project field-testing, snd attendance at either the 1967 
or 1968 English Project summer training institute. Of the 28, 10 had six 
to ten years of experience, 10 had less, eight had more. Nine hold the 
B.A., 14 the professional diploma, 3 the master's degree. (Two were 
unreported.) Twelve had participated in the program last year; 18 had 
attended one of the two teacher training institutes. (See Appendix 8.) 

C. PROGtlAM VARIABLES 

X 

1 . Goals and objectives 

The goal of the Language Skills staff is to devise learning tools for use 
in individualizing programs of instruction which will take pupils to the 
sixth-grade ability level in the graphic and oral representations of 
language and at the same time help them become self-directed learners. 

Curriculum planners are developing a continuum of objectives in reading, 
writing, and listening/ speaking that are sequenced from simple to complex. 
A variety of materials with optional modes of presentation leads to these 
objectives. These tools give the teacher a wide range of alternatives for 
guiding each child's leai'ning in the most appropriate manner. 

Self-directed learning is supported by an environment in which children 
engage in independent study usirg self* Instructional materials, wor in 
groups of two to four tutoring cacn other and checking each other's 
performance, ^et their own goals, and keep their own performance records. 



^Acknowledgement is made of the coi. . . j jution of the Language Skills planners, 
'n particular to Gerald Dykstra,and Aiin Port, in stating the program goals, 
tlves, and assumpti- ns , 
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The classroom is arranged to acconimodate the whole range of learning 
activities and tools* Instead of rows of desks, there are stations 
equipped for reading, listening to records and tapes, writing, type- 
writing, working with card stacks and Langusge Masters. (See Section 3 be- 
low, Component Units in the Language Skills Subprogram, for descriptions 
of the devices and materials provided in the stations.) Tables and 
chairs are so arranged that pairs or groups of children can work 
together. A sample floor plan is shown in Appendix 9. 

No stipulation is made as to the time allowed for a child to achieve 
sixth-grade competence in language skills. Once a child reaches this 
level, he continues to advance in language skills while studying litera- 
ture, language systems, and other school subjects. 

Behavioral objectives for each component of the program are presented 
in the description of component units found in Appendix 10. The 
Teacher’s Manual lists entry requirements for each component, activities 
for reaching the objective, procedures and criteria for measuring 
successful accomplishment (usually 90 to 100 per cent on the criterion 
measure), and guides foi alternate or subsequent programs. In most 
schools a quarterly progress report was sent home to parents. It states 
components in terras of objective outcomes and tells parents which pro- 
grams the child was diagnosed as not needing, which ones he is working 
on, and which ones he has completed, A quarterly student rating by 
classroom teachers on nine skills related to self-directed learning is 
also a part of this progress report. A sample report card is attached 
as Appendix 11 of this document. 

The Language Skills program is more than a collection of individualized 
materials; it is a comprehensive and coherent learning ’’system." (See 
Appendix 12 for a schematic representation.) It embodies an innovative 
philosophy of education incorporating new concepts of classroom organi- 
zation and grouping, as well as teacher roles. During 1968-69 most of 
the classrooms p 'ticipating in the program contained both kindergarten 
and first-grade students. Next year many of these classes will include 
second-graders as well, thus providing a richer opportunity for more 
experienced students to help the less experienced and serve as models 
for them to imitate. The tutors will benefit in turn from conununi eating 
what they have learned. 

Planners' projections of student progress in reading, handwriting, typing, 
and listening/speaking are listed in Appendix 13 for both kindergarten 
and first-grade students. together with a discussion of actual outcomes- 
These projections, made early in the second semester, show the minimal 
level of progress expected for 95 per cent of the students, for 50 per 
cent, and for the top 5 per cent. 



2. Program assumptio ns 

Certain assumptions underlie the philosophy of the Language Skills 
Subprogram, Among the most essential are these five; 
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a. Children differ in background, in interests, in styles of learning, 
in aptitude and speed, in thresholds for boredom, in educational 
needs, in need for indications of success, and in need to partici- 
pate in decisions affecting their own activities. 

b. Each child's needs, abilities, and interests must be considered if 
he is to acquire a proficiency in language skills. 

c. It is necessary to individualize instruction in accordance with 
each child's previous achievement, rate of learning, style of 
learning, and interest. 

d. The teacher's roles in the program are to 1) organize and manage the 
learning environment; 2) diagnose each child; 3) facilitate self- 
directed learning by encouraging participation in the selection of 
goals, materials, end modes; 4) maximize opportunities for success 
by guiding pupil selections; 5) monitor the progress of each child; 
and 6) serve as a model of the scholar at work. Conversely, her 
role does not include lecturing or disseminating infcrmation to be 
learned. 

e. Under proper supervision, children can learn effectively from other 
children in the class. 

3 . Component units in the Language Skills subprogram during 1968-69 

Appendix 10 lists and describes the specific objectives and materials 
used in the program. In the reading program, spindled or bound stacks 
of cards showing pairs of letters, words, phrases, and numbers are 
used by pupils to learn to discriminate differences between written 
symbols. Sets of Language Master cards match sound with symbol: the 
child slides the card into a slot in a machine which plays the word, 
phrase, or sentence shown on the card. A library of 160 books graded 
according to difficulty into twenty levels permits the child to move 
along to more fluency in reading. Other kits of books (Basic Reading 
Series, Science Research Associates sets) Intersperse readings with 
compreliension checks. 

In writing, sets of cards with flocked letters to trace with the 
finger, laminated books of models to copy, and film loops for use in a 
technicolor projector are available for the child to use. 

Books in the typewriting series move from single letter models to 
full paragraphs and eventually to free composition. 

The Language Master is the rriDinstay of the early listening and speaking 
components. Sets of sight-and-sound cards help dialect-speaking chil- 
dren to hear and produce English sounds that are difficult for them. 

The cards are also used to teach shapes, colors, and the alphabet. 
Grammar games, songs, and tapes of books from the reading collection 
are also provided. 
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4. 



Program activities of students 



To get some idea of how children spend their time in the Language 
Skills subprogram, where learning tools are supplied but activities 
are more or less freely selected, the evaluation staff monitored 
classrooms at Kalihi-Uka School between March and June, recording 
the amount of time pupils spent in different kinds of activities, 
(Procedures used and findings are reported in detail in Appendix 14.) 
Conditions were set which, if met, were presumed to validate the 
claim for individualization of instruction in Language Skills: first, 

that the pupil spends most of his time in study alone or with a part- 
ner, relatively little in total class activities; second, that suffi- 
cient options are available to meet his needs and interests; and third, 
that he has the option (especially early in kindergarten) of doing 
things other than language arts if he prefers. 

It was found that during the two-hour language arts period children 
spent 41.3 per cent of their time in independent instruction (using 
cards, books, and equipment); 19,7 per cent in pupil-pupil activity 
(working together on cards, games, books, and the like); 5.5 per cent 
with the teacher; 10,5 per cent in small groups (using the record 
player, tape recorder, books, and games); 1,4 per cent in total class 
activity; and 21,6 per cent in activities not part of language arts 
(such as wandering about, talking, painting, playing). In summary, 
pupils spent more than 7S per cent of their time working on language 
arts activities -- alone, in pairs, with the teacher, or in small 
groups --.and only a tiny portion in total class activity. These 
findings substantiate the claim that the Language Skills subprogram 
meets the conditions set for an individualized course of study. The 
freedom not to work on skills at all -•• an intrinsic part of the self- 
directed learning concept -- was also exercised about a fifth of the 
time. 

5 , Curriculum revis i on activities 

For a program still in the development stage, mor.3 effort goes iuto 
getting data for refining or revising materials than into judgi".^ how 
effective the product is. Before school began, curriculum pl?v.r,trs 
put to the evaluation specialist the questions that needed to 
answered so t)-at the planners could revise the components they wfre 
working on. They needed to know which modes worked well and which did 
not, which sustained interest, which seemed too easy or too difficult. 
They needed f igments on the sequencing of the materials, on the 
effcctivenes of differing modes, and dozens of other matters. The 
evaluation specialist devised procedures and forms for collecting, 
analyzing, and reporting the needed data. Appendix 15 describes the 
system and shows samples of the record forms used. 

6. Percejptions of the program 



An instructional program is no better than it is perceived to be, 
regardless of its intrinsic merit, because negative attitudes can blunt 
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or block its effect. If teachers, parents, or principals are negative 
toward it, their feelings can rub off on the children. The attitudes 
and reactions of outside observers, especially knowledgeable and per- 
ceptive ones, also are useful as clues to deficiencies calling for 
attention, For these reasons the perceptions of persons in or related 
to the Language Skills program are important. The responses of stu- 
dents, teachers, data collectors, principals, parents, and visitors 
were sought and examined. 

a. Students 

Responses to the Language Skills subprogram from foity randomly 
sampled students interviewed individually at Kalihi-Uka and the 
Laboratory School were quite positive, All the children liked 
school, and 31 gave answers that showed a particular preference 
for language arts activities. Thirty-eight liked the freedom to 
select their own programs and 29 were actually doing it. Thirty- 
eight liked to tutor other children while 34 said that they liked 
to have another child tutor them. Fifteen children were able to 
see and tell the purpose of specific activities. All but one of 
the forty felt they were doing well in Language Skills -- an 
important finding, since a sense of success frequently predisposes 
a child to successful accomplishment. (See Appendix 16 for a 
fuller report,) 

b. Teachers 

Teachers* perceptions of the program were communicated casually 
through notes and comments to planners and evaluators and in 
discussions during bi-monthly meetings between planners and Kalihi- 
Uka teachers. Reactions were solicited more formally at an evalua- 
tion workshop in late May and through a Q-sort for assessing atti- 
tudes. The planners shared teachers* reactions and suggestions at 
the planners* weekly meeting and considered them as they developed 
or revised materials. 

Written comments from Molokai teachers, often channeled to planners 
by the District Coordinator, noted such matters as the need for 
additional copies of materials, prevailing practice among pupils in 
classrooms, materials or procedures needing attention of some kind, 
and recommendations for filling apparent gaps. Teachers commented, 
in somewhat more detail, on the same matters when planners or eval- 
uators visited the classes. Their oral comments gave valuable 
clues to their perceptions of the entire Language Skills subprogram 
The general consensus of enthusiasm is sunoriarized in one teacher* s 
comment that she wouldn't want to go back to the old ways and 
that although she found the new program more tiring, she had a 
greater sense of accomplishment at the end of each day. 

The workshop held on May 24 for evaluating the 1968-69 field trials 
and soliciting reactions to plans for next year was attended by 
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25 teachers and 15 Curriculum Center staff members. A general 
briefing on evaluation was followed by small workshop sessions 
in which the following topics were taken up: reading; writing; 

listening/speaking; typewriting; classroom organization and 
record keeping; potential and problems of peer teaching; evalu- 
ating the evaluation; teacher training; observing individualized 
instruction; and inproving communication among teachers, planners, 
evaluators, and administrators. A summary of comments and sugges- 
tions under each of these headings is attached as Appendix 17. 

The sessions were fruitful for both teachers and planners. The 
teachers welcomed the opportunity to talk things over, and the 
planners came away with some very useful recommendations. 

The Q-sort technique was used to analyze and compare the attitudes 
of 24 teachers and nine planners. They were asked to sort 48 
statement cards into nine categories ranging from ’’strongly agree” 
to "strongly disagree." The statements had to do with individu- 
alized instruction, curriculum content, classroom activities 
and organization, peer teaching, the teacher’s role and training, 
the nature of the learner, and outside influences on the curric- 
ulum. The purpose was to find out teachers’ attitudes and to 
discover discrepancies between the responses of teachers and 
planners as a clue to planning the six-week summer workshop for 
training teachers. The results (Appendix B) showed high congruence 
of attitude between the two groups. Teachers and planners both 
recognize variations in interest and aptitude for language skills 
among children and agree that a sense of personal worth is needed 
if a child is to progress. Both groups doubted the value of tradi- 
tional letter grades in individualized instructional programs. 

There was less unanimity among the teachers than among the planners, 
who had together arrived at a consensus in the process of designing 
the curriculum and preparing the materials. 

c. Data collectors 

Eight mothers were enployed to record data for two hours each day 
in the eight classrooms at Kalihi-Uka School. For two of them, 
this was the first experience working with children in school. All 
eight were interviewed to discover hew they liked the work, what 
aspects of it they found rewarding or frustrating, and what they 
thought of the Language Skills subprogram, Iheir responses to the 
ten questions asked are described in Appendix 19. All enjoyed 
working with children and seeing them progress. They found their 
task uninteresting only when children became impatient, bored, 
frustrated, or restless at their work. This happened when the 
children were obliged to read something a second or third time or 
to go over material they had not yet mastered, 

d, Principals 

In late April and early Nay the evaluation specialist interviewed 




IH 



14 



the principals of the six schools to discover their opinions 
about the program, to get iiieir suggestions for improving it, 
and to find out their views of their roles in the project. The 
principals favored the program and spoke freely of both its posi- 
tive and negative effects on students, teachers, and the total 
school program. They shared comments of students, teachers, 
parents, emd visitors and made many suggestions for improving the 
program and disseminating it to the community. Among the strengths 
reported by the principals were the childrens* progress in lang- 
uage use and in self-reliance. Four of them felt that the record- 
keeping demands were excessive and two thought that the report 
cards were overly complicated. Appendix 20 contains a full report 
of the principals* responses to the questions. 

e. Parents 




To get some notion of parents* awareness of and response to the 
Language Skills subprogram, an evaluation specialist interviewed 
ten parents of randomly-sampled children in the program, eight at 
Kalihi-Uka and two at the Laborator>* School. Nine parents thought 
that the program was helping their children. Some cited specific 
evidence, such as the child's taking initiative in getting books to 
read. Without exception, the parents thought it a good idea to 
begin reading, writing, and typing in kindergarten. Eight favored 
the peer-teaching idea, although two of them doubted the tutor's 
competence and questioned whether tutoring would impede the learning 
of the tutor. Four parents felt that the report card was hard to 
understand. (See Appendix 21 for summary of parents* responses.) 

f. Visitors 



The remarks and judgments of visitors are useful for evaluation 
because they foreshadow the response of the public and the pro- 
fession. Furthermore, their questions point to matters which must 
be attended to in publicizing and implementing the program. 

A visitor log kept by District Coordinators and secretaries shows 
that over 600 persons visited the project headquarters or field- 
school classrooms between January and May. Visitor guides kept ^ 
record of questions and comments made by visitors. (See Appendix 
3.) In addition, over a hundred visitors completed, a questionnaire 
and returned it by mall to the evaluation section after their visit. 



The questionnaire asked which aspects of the curriculum they found 
desirable and undesirable and which ones they would be interested 
in initiating. The individualization of instruction and the vari- 
ety and adequacy of the materials were singled out for praise by a 
large number of the visitors. Some expressed concern about the 
volume of record keeping done by some teachers and the difficulty 
of monitoring the learning progress of children following different 
routines. Others questioned the rationale for specific emphases or 
pointed to gaps they considered crucial. The aspects of the curric- 
ulum they were interested in init ating covered a wide range; sundry 
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strategies and materials for individualizing instruction were 
most often mentioned. (See Appendix 22 for a complete record 
of responses.) 

g. Summary of perceptions of the Language Skills subprogram 

The reactions of all groups interviewed were generally positive. 
Students seemed to enjoy language arts; teachers expressed a 
sense of accomplishment j parents seemed satisfied that their 
children were learning; visitors were impressed with the program's 
variety and versatility. Two factors appear to require special 
attention: record-keeping and peer-tutoring. 

7. Materials and equipment 

Materials and equipment were evaluated on the basis of student prefer- 
ences, ratings by teachers and planners, use by students, time required 
to complete sequences, and amount of repair needed. 

a. Student preferences 

An inventory of what modes children liked and disliked was adminis- 
tered to all students in those classrooms having the following: 
laminated i.’riting books, Language Master, card stacks, film loop 
projector, flocked cards, reading books, and typewriter. (However, 
certain classrooms were without the film loop projector at the time 
of the survey.) 

Some generalizations can be made on the basis of the inventory. 

The most favored media were the typewriter and the card stacks; 
least favored were the flocked cards, the film loop projector, 
and the Language Master. The rankings vary significantly from one 
school to another. The flocked card mode was the only one to vary 
significantly by grade level. It appealed to some kindergarteners 
and a few first-graders, but not to second-graders. (See Appendix 
23 foi further information and data.) 

A second survey was made two weeks after the first. A total sample 
of 40 children from Kalihi-Uka and the Laboratory School were asked 
to separate picture cards of seven kinds of media into two piles, 
those they liked and those they disliked. Although many had changed 
their minds during the interval, there was a high correlation between 
the most liked on the first survey and those liked on the second sur- 
vey. Similarly, there was a high correlation between the least liked 
on the first survey and those disliked on the second survey. The 
reasons the children gave for their likes or dislikes most often had 
to do with their sense of success or failure, less often with whether 
or not the unit was fun to use. 

b. Teacher and planner equipment preferences 

The preferences of teachers and planners were ascertained through an 




16 



20 



equipment purchase simulation. They were asked to allocate 100 
chips to the purchase of eight equipment items of varying chip 
values for kindergarten classes in the way they felt would con"^l- 
bute most to accomplishing the objectives of the Language Skills 
program. The number of units of each kind requested ty Kalihi-Uka 
and Molokai teachers and by the planners is shown in Appendix 24. 

The data show that teachers would prefer fewer reading book sets, 
typewriters» film loop projectors, and tape recorders and more 
Language Masters and record players than the planners would. 

It is interesting to note that the ^^olokai teachers* preference 
for the Language Master is paralleled by the pupils* high ranking 
of it, whereas at Kalihi-Uka teachers and students both favored 
the card stacks. However, no cause-effect conclusions can be 
draivn from the data. Perhaps teacher and student attitudes were 
mutually reinforcing. 

Teachers taking part in the simulation were asked to answer four 
questions, three of which had to do specifically with the use of 
the Language Master. There was wide disagreement about its appeal 
and usefulness. Suggestions for iniprovement were very specific, 
having to do with technical matters (quality of voice reproduction, 
repair service), pedagogical matters (use of words and pictures 
more familiar to children, ways of eliciting responses), and practi- 
cal matters (size and number of cards, ways of checking children’s 
progress). Nearly all these suggestions are being acted upon by 
planners, (See Appendix 25 for suiimary of responses.) 

c. Teawher rating scale for Language Skills materials 

A rating scale was devised to deteimine the teachers' judgment 
concerning the frequency of use and the suitability of individual 
materials for meeting the objectives of the skills subprogram. 

These data were collected to provide one type of evidence useful 
in streamlining the curriculum package. Teachers were asked to 
rank specific items in the reading, writing, and listening/speaking 
programs in three levels of frequency of use, and four levels of 
suitability. Where they gave an item the lowest ranking, they were 
asked to tell why. The results of the survey were tabulated for 
use by the project manager and planners, (See Appendix 26.) 

d. Time required for completion of selected language Skills sets of 
materials 

Several teachers regularly recorded the dates their pupils entered 
and completed each program. At the end of the year the records 
for one three-on-two classroom^ on Molokai and one self-contained 
classroom at Kalihi-Uka were analyzed. The completion times in 
numbers of school days from entry to completion of selected materials 
were tabulated. 



^A classroom in which three teachers take care of two classes that combine 
^re than a single grade level. 
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The records do not, however, shov; ^ '^ojount of lime a child spent 
on the materials each day, the numuci of other programs he was 
working on at the time, nor the days he was absent. Likewise, 
there is little data on beginning programs and advanced programs, 
since fewer pupi^.s used them, 

Il.e summary shows a tremendous range in the length of time required 
for students to compl'ite each program, TYiis evidence tends to 
substantiate the need for programs that are adaptable t<i individual 
learning r^tes and styles. The data also confirm the obseivation 
of planners that as children progress through a sequenced program 
to sets of approximately the same quantity at higher levels, the 
time they need to complete a set decreases, (See Appendix 27.) 

This indicates that students gain speed as they advance through 
the materials, 

e, Equipruent repair record 

Six types of data were collected on the repair of equipment used 
in the Language Skills classrooms: the number of units of each 
type in use in the field schools; the number of units needing 
repair; the number repaired at the school, at the Curriculum Center, 
and by an outside agency; the average repair cost; the most fre- 
quently needed repairs; and the average length of time the equipment 
was out of use for repairs. These data were collected to provide 
project administrators with a picture of the reliability of various 
types of equipment so that they could determine which ones perform 
acceptably and what changes need to be made in purchasing, modifying, 
maintaining, and repairing the units, (See Appendix 28 for speci- 
fic findings,) 

The Curriculum Center's media specialist considered the breakdown 
rate excessive for che Language Master and the film loop projector, 
but acceptable for the other types of equipment. Plans for next 
year call for the purchase of new Language Masters, Super 8 film 
loop projectors will replace the standard 8 mm projector. Instruc- 
tion on media use and maintenance is being given to teachers in the 
summer training program with the expectation of reducing the nunber 
of units sent in for repair. 

Modifications in the equipment repair procedures are being consi- 
dered which will permit schools to send equipment directly to a 
repair agency in order to expedite its return to the classroom. 
Service agreements with the vendor are also leing explored, 

D. PROTOTYPE OUTCXIME VARIABLES 



1 , Student perfo m .ance in language skills 




Sixteen language skills programs wen selected for systematic monitoring 
this ycari Appendix 29 lists these programs together with the number 
of kindergarten, first, and second-grade children for whom report card 
data were available. In most cases there were data on 224 kindergarteners, 
241 first graders, and 34 second graders -- that is, 93 per cent of the 
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children in the program. This figure encompasses all types of children, 
including those with learning disabilities. The other 7 per cent were 
those who transferred out of the program during the last grading period 
or whose report cards were missing. Several programs, such as type- 
writing, were not supplied in certain classrooms; hence the percentages 
for these are based on fewer students. 

Since Language Skills is an individualized instructional program, it 
was decided that the predicting and reporting of group progress in 
mean performance terms was insufficient. Instead, a range of expecta- 
tions of performance was projected. 

Children's progress in skills was then measured by comparing the planners' 
expectations at the beginning of the second semester with actual out- 
comes by June 1969. For each of the language skills, three levels of 
expected achievement for kindergarten and for first grades were projected 
by the planners -- a minimal level (to be achieved by 95 per cent of the 
children); an average level (to be achieved by 50 per cent of the chil- 
dren); and a high performance level (to be achi(ved by 5 per cent of t\e 
children). (See Appendix 13) 

Chart 1 compares curriculum planners* projections with students’ achieve- 
ment in language skills. The figures in parentheses following each 
projection show the actual outcomes. For exajaple, while it was pro- 
jected that 95 per cent of all kindergarten children in the program would 
be able to discriminate between letters of the alphabet by the end of 
the school yeaii the data revealed that 100 per cent of the kinder^^arteners 
achieved this objective. 

In considering the 24 projections for k:ndergarten and first grade stu- 
dents for all four areas of the Language Skills subprogram, there were 
five cases where actual performance exceeded projections by more than 
10 per cent and only one case (the program for dialect speakers) where 
the performance was more than 10 per cent below the projection. In 
this case it was found that some of the students recorded as not having 
started the program were not in need of it, although some teachers did 
not record this distinction. Tliese data indicate that the Language 
Skills subprogram for 1968-69 has met the expectations of the curric- 
ulum planners. The actual outcomes recorded will serve as baseline 
data for next year. 

2 . St udent perfonnance in self-directed learning skills 

The progress of Kalihi-Uka kindergarten and first-grade children in 
directing their own learning was assessed by comparing teachers' ratings 
during the first and fourth quarters of the school year (See Appendix 
30). Pupils were rated on their ability to plan an activity, to change 
activities without teacher direction, to wrk independently, to gather 
the materials needed, to read their names and find their stickers in 
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CHART 1 



Comparisons of Curriculum Planners* Projections with Actual Outcomes 
for Kindergarten and First'^Grade Student Achievement in the Language 
Skills Program by June 1969* 
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Figures in parentheses refer to the actual outcomes by kindergarten or first 
grade children. 
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a program^ to put their naterials away after using them, to accept 
tutoring from other children, to help others leam, and to mark 
their own progress in their record folders. 

The students improved most in planning their own activities. They 
progressed perceptibly in changing activities without teacher help, 
in reading their names and finding their stickers in programs, and in 
marking their progress in their record folders. The least gain was 
recorded in willingness to accept tutoring from other children. This 
finding suggests that readiness to accept help may be a function of a 
child’s personality or value system rather than of classroom training 
and that new ways may need to be found for encouraging children to 
receive assistance from each other. 

An analysis of teacher ratings on self-directed learning skills during 
the fourth quarter shows that girls scored somewhat higher than boys, 
first graders higher than kindergarteners. The single exception is 
in willingness to be helped by other children. Kindergarteners are 
apparently less inhibited about receiving help from peers than are 
first graders. This evidence implies that first giaders may be 
unwilling to accept tutoring from those they consider to be their 
equals or inferiors, if so, it gives a clue for improving the peer- 
tutoring practice next year. 

3 . C ase studies 

The case study technique was used to develop hypotheses about how 
different kinds of children perform in the skills programs and how 
they are affected by them, to identify variables that might correlate 
with success or failure in the program, and to explore variables that 
should be included in a longitudinal stud/ of the program. 

Subjects for the year-long study were ■^'“"‘d'mly selected from the eight 
K- 1 classrooris at Kalihi-Uka in such a ’.r r that there was one pupil 
from each classroom and one in each of eight categories; dialect 
speakers with high, medium, and low IQs^ standard s’^eakers with high, 
medium, and low IQs; a mentally retarded child; and an emotionally 
disturbed child. Mental retardation, as defined for this study, is 
an IQ of 69 or below. An ejnotionally disturbed child was one who 
htd been so diagnosed by the school psychologist or guidance counselor. 
Ranges for high, medium, and low IQs were taken from the classifications 
listed in the 1960 revision of the Stanford-Binet manual: high - 110- 

119; medium = 90-109; low - 80-89. 



Stickers arc adhesive name tags the children use to mark their place in 
the materials they are working when they finish a component they put the 
sticker in their record folder to indicate that they have completed that 
component. 
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Several months after the case studies were initiated, it was decided 
to include in the study one child who made very rapid progress and one 
who worked on the program under parental duress* These latter two were 
from a K-2 class and a Head Start class in the Laboratory School, 

Data were gathered from records and observations. Teachers in each of 
the ten classes helped by completing questionnaires and interviews 
about the children’s social, emotional, and academic background. Infor- 
mation about their home life and physical health was obtained from the 
cumulative files and the guidance counselor. Questions about children's 
behavior patterns were answered through interviews with teachers in 
alternate months. A data collector observed each child several times, 
recording his activities in class. Attitudes were judged from the 
children’s responses in an interview. School report cards were the source 
of data for the number of programs completed, progress in certain key 
programs, and growth in self-directed learning. Finally, teachers were 
asked to indicate significant changes in behavior during the year. Con- 
densed syntheses of the information collected from the ten case studies 
"ire attached as Appendix 31, 

From the case studies there emerged seven hypotheses for fur'.her investi- 
gation: 

a. A substantial portion of the variation in progress is accounted for 
by such teacher variables as attitude toward the programs, the 
degree to which she stresses student choice, and the amount of atten- 
tion she gives to individual children. 

b. Standard IQ test measurements are less effective as predictors of 
student success in this program than they are in traditional language 
skills programs, Ktore effective predictors should be sought. 

c. The range of student performance in language skills is greater within 
a single grade level than is the range between grade levels. If 
this finding is substantiated it will supp rt the decision to com- 
bine students from several grade levels within the same classroom, 

d. Student success in tutoring will be iinproved by better instruction 
in the how and why of tutoring and by allowing tutors more choice in 
whom they tutor. 

c. A nore precise system should be developed for measuring and comparing 
achievement in language skills and in self-directed learning skills. 

f. There is no significant correlation between programs childrci: say 
they enjoy and programs in which they participate most frequently or 
make most progress. 
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g. Data collected from case studies will assist curriculum planners 
in better understanding the effects of programs on children only 
if frequent interchange of information occurs between the data 
collectors and the planners. (Much of the value of the case 
studies this year was lost by not providing timely information 
to the planners.) 

4, Annual performance tests 

Review tests in reading, writing, typewriting, and listening/ speahlng, 
were used to discover what students had mastered during the school 
year. Everyone took the writing test; the other three tests were 
administered to a random sample of students in the program. 

a. Reading 

Reading ability was assessed by using a special stack of 50 cards -- 
10 for discriminating pairs of words, 8 for large letters and 8 
for small letters, 16 for reading individual words, and 8 for reading 
sentences. Thirty-two Kalihi-Uka children -- a kindergarten boy 
and girl and a first-grade boy and girl from each of the eight class- 
rooms -- took the reading stack test. Their scores are summarized 
in Appendix 32. 

On word discrimination and large letters, kindergarteners did about 
as well as first graders. On the other objectives, first graders 
scored higher, with girls doing better than boys on the more diffi- 
cult cards. Of the 16 first graders, five could read none of the 
eight sentences without making errors. However, six of the 16 
kindergarteners were able to read at least one sentence without 
making errors. 

b. Writing 

Three parallel forms of a short test booklet were used to measuie 
progress in writing numbers, letters, letter combinations, words, 
and short sentences. It should be noted that, except in one class, 
children were taught cursive writing but not manuscript. Still, 
since ioany had picked up rmnuscrlpt one way or another, they were 
permitted to write their names either way on the test booklets. 
Preferences in using cursive writing went from only 15 per cent in 
kindergarten to 42 per cent in first grade to 63 per cent in 
second grade. Preference for manuscript went from 44 per cent in 
kindergarten to 26 per cent in first grade and 20 per cent in 
second grade. The Incidence of mixed writing went down from 29 per 
cent in kindergarten to 23 per cent in first grade to 10 per cent 
in second grade. 

In nearly all cases, girls made fewer errors in copying than boys 
did. The rvsre difficult the task, the bigger the gap became. 

In copying a four-word sentence in cursive style correctly, 50 per 
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cent of kindergarten children, 83 per cent of first graders, and 
95 per cent of second graders were successful. 

Data from the handwriting test r :e being studied carefully for 
clues to better sequencin^i of tasks within the writing materials. 
Analysis of data by classrooms should show whether or not certain 
programs (for exainple, the trace-and-write set on film loops used 
in some classes) make a difference. The confusion of cursive and 
manuscript styles in kindergarten and grade one raises some ques- 
tions for the planners to consider. 

c. Listening/Speaking 

In may, 240 students were part of a study designed to compare the 
sound-discriminating ability of younger and older children, of 
dialect speakers and standard speakers, and of students in the 
Sounds of English program and those not in the program. Forty 
students were randomly selected in each of grades kindergarten, 1, 

3, and 5 at Kalihi«Uka School. A control group of 40 kindergarten 
pupils and 40 first graders was randomly selected at Kaiulani School, 
where the student body is comparable to Kalihi-Uka*s in ability and 
background. 

Sixty-five pairs of words exhibiting a subtle sound distinction 
were used, 33 pairs in one test, 52 in the other. Tests took the 
form of sets of picture cards, each card having a pair of illustra- 
tions. After the test administrator identified the two pictures 
(a sheep and a ship, fot example) the student was asked to point 
to the one representing the word the tester then pronounced. The 
tests were given individually. 

Two hypotheses were tested: (1) that the higher the grade level, 

the fewer errors there would be, and (2) that K-1 children in the 
program would make fewer mistakes than similar children not in the 
program. The purpose of testing the first hypothesis was to dis- 
cover how much children's ability to distinguish the sounds of 
English improves, without special attention as they grow up in an 
English-speaking environment. The reason for testing the second 
was, of course, to find out whether the Sounds of English program 
had significant impact cr not. 

The performance of all children tested was coinpared by grade level, 
and the performance of kindergarteners and first graders in the 
two schools was compared. Errors were tabulated for each of the 
65 pairs of words. Test results on 12 kindergarten and 52 first- 
grade children at Xalihi-Uka diagnosed as dialect speakers were 
examined to identify the pairs of sounds they most often confused 
so that these could be given special emphasis in the Dialect 
Markers program for children whose first language is not English. 

All data are displayed in the tables shown in Appendix 33. 
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In summary, total errors made by children in grades kindergarten, 1, 

3, and S were 211, 138, 44, and 19 respectively, validating the 
hypothesis that errors in hearing would decrease at progressively 
higher grade levels, l^he comparison between scores of partl.cipating 
and non participating children revealed highly significant differences 
in listening; whereas kindergarteners at Kaiulani made 211 errors 
and first graders 138 errors, their counterparts in the program at 
Kalihi-Uka made 114 and 59 errors, 

There were fifteen minimal pairs for which 10 per cent or more of 
the nou-participants could not hear distinguishes: fin/thin, tin/thin, 

they/day, sledge/sleds, tug/tugged, mouse/mouth, lass/lash, bat/bath, 
choke/joke, rake/wake, match/mats, pool/pull, sick/thick, sock/socked, 
and box/boxed. The pairs which gave dialect speakers significantly 
more trouble than standard speakers were tug/tugged, bat/bath, 
treat/tweet, they/day, box/boxed, match/mats, and sledge/sleds. 

This study produced data for ranking the minimal contrast pairs for 
difficulty and for eliminating those missed by few K-1 children and 
none at higher grade levels. 

d. Typewriting 

To provide information helpful in evaluating the objectives of the 
typewriting lessons (particularly correct fingering), a study was 
designed to determine for kindergarten and first-grade children the 
numbers and kinds of ooth fingering errors and typing errors and 
the amount of time needed to type a lesson. Six criterion lessons 
were selected in which one, two, or three rows of keys were used 
for small letters and for big letters. Five kindergarteners and 
five first graders who had completed a particular typing sequence 
were tested on each criterion lesson. 

First graders excelled kindergarteners in correct fingering, accu- 
racy, and speed, averaging only one fingering and one typing error 
per criterion lesson page. However, the number of typewriting pro- 
grams completed showed no correlation with a child's age. As chil- 
dren advanced to more difficult exercises, their fingering errors 
did not increase. 

In suBuTPry, the criterion tests demonstrated that kindergarten and 
first grade children can not only learn to type from self-instruc- 
tional lessons, but that they can leam to do it accurately. 

5 , Peer teaching outcomes 

Some classroom visitors observing peer-teaching dyads detected such 
faulty practices by tutors as allowing errors to slip by unnoticed or 
providing answers before the learner had time to respond. These obser- 
vations prompted studies to find out how much misteaching was actually 
going on. 
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In the first study, pairs of children using card stacks or new reading 
words were monitored unobtrusively in four Kalihi-Uka classrooms. Of 
the tutors, 16 were kindergarten pupils, 29 were first graders. Observ- 
ers watched each dyad for ten minutes, recording the tutor’s grade and 
seXi the number of cards attempted, the number of learner errors made, 
the number of errors detected by the tutor, ana the deviations from 
established tutoring procedures. Appendix 34 displays the data. 

It was found that when the tutor was a first grader, almost twice as 
many cards were used in the dyad as when the tutor was a kindergartener. 
When first graders were tutoring, learner errors averaged 11 per cent, 
as against 18 per cent when kindergarteners were tutoring. These 
findings may be accounted for by the fact that first graders were most 
likely to be helping more advanced learners than the kindergarteners 
were. Kindergarten tutors let 42 per cent of the errors pass unnoticed, 
as against 20 per cent for first graders. The percentages are high; 
yet if undetected errors are calculated against the total number of 
cards used, the rates are only S per cent and 2 per cent. Forty- four 
per cent of kindergarten tutors and 52 per cent of first-grade tutors 
caught all errors. In other words, nearly half of the tutors are doing 
an outstanding job. For the other half, something needs to be done -- 
better training, more supervision, or more careful screening. 

About half of the time the tutors failed to follow instructions to 
take the learner back to the nearest red review card when lie missed a 
blue card. The planners did not consider this a serious mispractice. 

This fact suggests that perhaps going back to the red card is not really 
necessary. 

The second peer-teaching study, conducted with a sample of ten dyads 
in two classrooms at Kalihi-Uka School, was designed to determine whether 
tutoring enhanced the tutors' mastery of the material, The results 
(Appendix 35) show that tutors averaged 11 errors on 30 cards before 
tutoring, but only four after tutoring. A week later, they averaged only 
three errors on a retention test. The conclusion seems valid that for 
most children teaching is an effective way of learning. This study 
should be replicated next year with a larger number of children and re- 
designed to overcome the problem of test-retest interaction. 

6. Comparative studies 

Several comparative studies were made at Kalihi-Uka School in an effort 
to get some measure of the effectiveness of English Project materials 
in helping children learn to read. In the first study a sample of 
second grade students, none of whom had had the skills program, were 
compared with a sample of first grade children who have been in the 
program since kindergarten. Till rty-one students in each group were 
tested on reading selected words and sentences, some from the English 
Project materials and the rest from tradebooks. The first graderf 
did slightly better on both words and sentences than the second gracl^rs. 
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(See chart in Appendix 36) Of first graders, 19 could read some 
sentences^ as against 16 second graders, TVrelve children in grade 
one and 15 in grade two couldn’t read any of the sentences. On word 
recognition, 18 first graders and 15 second graders were able to read 
between five and 13 selected words; 13 first graders and 16 second 
graders read fewer than 5 words, 

A second study compared performance of kindergarten students partici- 
pating in the Skills program last year with the performance of partici- 
pating kindergarteners at the end of the first and second semesters 
this year. Records were examined to detennine what percentages of 
children had completed certa^'n components. In ail four reading skills 
(naming the capital letters and the small letters, recognizing like- 
nesses and differences in pairs of words, and beginning to read books), 
this year’s group did better after a half year than last year’s group 
did after a ful I year. In three of the four skills, this year’s group 
more than doubled the performance of last year's group. Since the 
kindergarten students this year are of approximately equal ability 
with last year’s, much of the marked difference in perforniance may be 
attributed to the unproved materials and procedures used this year. 

For example, this year, for the first time, kindergarten and first-grade 
students were combined in the same classrooms, so that kindergarteners 
were frequently tutored in the programs by the older children. Appendix 
37 shows a comparison of the two groups. 

In a third study the number of books read by first-grade pupils not in 
the program last year was compared with the number read by participating 
first graders this year. This year’s group read two to three times as 
many books as last year’s: the averages are 13 books for this year’s 

first graders versus five for last year’s. And whereas no students last 
year read 15 or more books, nearly two-fifths of this year’s class did. 
Close to a fifth of them read at least 25 books. Figures are shown in 
Appendix 38. 

7 . Correlation of selected variables affecting the ^roy^ram 

Data for the approximately 430 students who participated in the Language 
Skills subprogram this year were keypunched and analyzed using an IBM 
360/55 computer with computer programs from the Biomedical Library 
series (BM)) . Three tj'pes of data were used: (1) background information 

on the child (sex, IQ, and socio-economic status), (2) information on 
the child’s ability to carry on certain classroom activities such as 
gathering materials needed for a particular lesson, and (3) records of 
the highest level the child achieved during the year among those programs 
in reading, writing, typewriting, and listening/speaking whicn were 
monitored for evaluative purposes. 

The student population was divided almost equally between boys and girls. 
Intelligence was in the normal range; socio-economic status was below the 
United States average. 

Although all possible combinations of these data were used in the corre- 
lation study, only those correlations which are of particular interest 
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are reported here. As expected, the correlations between sex and IQ 
and between sex and socio-economic status (SES) were essentially zero. 
The correlation between IQ and SES was .45, a somewhat lower correla- 
tion than is usually found between these two variables. 

Correlations between sex, IQ and SES and the highest levels achieved 
in reading, writing, typewriting, and listening/ speaking (see Table 1) 
were low, with the exception of moderate correlations between IQ and 
reading for first-grade students. These low correlations of sex, IQ 
and SES with language skills are quite important. Basically they 
indicate that success in the HCC Language Skills subprogram is not 
wholly determined by traits that a child cannot control. Thus, chil- 
dren of lower SES have about the same chance to succeed in the program 
as children from upper SES levels. In fact, students in the lower 
socio-economic level progressed through slightly more typewriting 
programs than children from the upper socio-economic level. 



TABLE 1 



Correlations Among Highest Language Skills Levels 
Addeved in Four Areas and Sex, IQ, and SES for 
Kindergarteners and First-Grade Students 

Reading Writing Typing Listening/Speaking 



Grade 


K 


1 


K 


1 


K 


1 


K 


1 


Sex* 


.04 


.23 


,27 


,19 


,24 


,28 


.03 


.07 


IQ 


,21 


.51 


-.11 


.08 


.24 


,22 


.35 


.23 


SES 


.18 


.27 


-.01 


-.03 - 


.16 


-.08 


.18 


.04 



♦Coded as Boys = 1, Girls = 2; thus the higher the 
correlation, the better the girls did in compari- 
son with the boys 



The low correlations among the four highest levels of achievement in 
reading, writing, typing, and listening/speaking (see Table 2) were 
of particular importance in that they indicated that it was not necessary 
that a child perform well in one skill area to be able to perform 
well in another; the individual areas were essentially independent. 
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TABLE 2 



Correlations Among Highest Achievement Levels in 
Four Areas of Language Skills 



Reading 


Writing 


Typing 


Listening/Speaking 


Reading — 


.33 


.37 


.48 


Writing 




,19 


.25 


Typing 






.14 


./Spkg. 






— 



Correlation of teachers* rating;, for 180 students at Kalihi«Uka 
on nine classroom activity skills during the first and fourth 
quarters of the school year showed moderately high correlations 
(*70, *69, ,68, and .65 respectively) on gathering needed materi- 
als, putting materials away when finished, helping other children 
learn effectively, and locating one's own sticker. The lowest 
correlation, although moderate (.49), was found in planning 
at leajst one activity daily. The mean difference gained between 
ratings on the first and fourth quarters was found to be highest 
on locating one's sticker and on going from one activity to another 
without teacher direction. Least stud'jnt gain was made on recording 
one's progress, working independently, and accepting tutoring from 
other children. This last skill was also the only one in which 
kindergarten children were rated higher than first graders. In 
general, these latter three skills seem stable and are not easily 
affected by instructional methods. 

Correlations between classroom skills and the four language skills 
for the fourth quarter were all positive except for accepting 
tutoring from other children, which showed a -.02 correlation with 
reading and a -.28 correlation with typing. This same trait corre- 
lated at only .01 with writing and .12 with listening/ speaking. 

This suggests that the ability to accept tutoring from other children 
is not related to success in language skills, while the other eight 
traits are positively related to success in language skills. The 
classroom activity skills correlated most highly with success in 
reading. These correlations ranged from .38 for putting materials 
away to .53 for working independently and .57 for helping others 
learn effectively. 

Graphs showing the correlation between highest reading level and IQ 
levels (See Appendix 39) indicate that students having read over 15 
books ranged in IQ from 85 to MS. Likewise, students having read 
over 35 bocks by the end of grade one came from all socio-economic 
levels, including those whose fathers had less than an eighth grade 
education and were in unskilled occupations (see Appendix 39). 
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Graphs have also been prepared that show an almost perfect coirelatlon 
between the test scores for first graders on the Gates-MacGinitie 
Reading Test and achievement within the reading program after students 
have begun to read books (See Appendix 40) . Before that time the 
correlation is quite low, since the early decoding programs are based 
upon pattern words rather than the most frequently used words in 
basal readers which serve as a basis for the Gates-MacGinitie test. 

This data partially validates the sequential order of the reading 
levels from easy to difficult. 

In summar>', the correlations of student success among the four language 
skills areas is relatively low with the exception of a moderate corre- 
lation between reading and listening/ speaking. This indicates that 
it is not necessary that a child perform well in one area to be able 
to perform well in another. 

The IQ scores correlate moderately with reading and with listening/ 
speaking, and only slightly with writing and typewriting. Sex and 
socio-economic status correlate only slightly with any language skills. 
The lack of high correlations between IQ and SES with success in the 
language skills is quite unusual; it is empirical evidence that the 
language skills subprogram is capable of functioning effectively even 
with children from low IQ and socio-economic levels. 

Positive correlations among eight of the nine classroom activity 
skills and success in the four language skills indicate the compati- 
bility between the individualized instructional goals and the langua^ 
skills goals. The high positive correlations between scores on th^ 
Gates-MacGinitie Reading Test and achievement in the reading progra 
beyond the early decoding stage support the sequential order of booV 
in the reading levels from easy to difficult. 
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IV. THE LITERA'fURE SUBPROGRAM^ 



A. DEVELOPMENT OF THE CONCEPT OF EVALUATION 1968-69 

The behavioral framework is not sufficient for defining objectives and 
assessing progress in the literature subprogram. The foremost goal of 
the program is to foster and develop the child's response to literature. 
But "response" is not necessarily observable, nor can its form be pre- 
dicted because, initially at least, it is personal, impressionistic, 
and based on a value system that is implicit but nonetheless real to 
the child. Hence mechanistic stimulus-response measures for evaluation 
will not do; more fitting ones must be invented. 

But the complexities of defining response to literature in terms amenable 
to measurement are formidable. For one thing, there are many t>pes of 
response (physical, emotional, intellectual) which some (Krathwohl, for 
example) may regard as inherently "higher" or "lower" than others. And 
there are levels or gradations of response, but these have to do with 
the sophistication of the form the response takes or with the complexi- 
ty of the literary work evoking it. 

Because of these unresolved difficulties, it was decided to put major 
emphasis in this year's evaluation on testing the content of the units 
and, concurrently, to work toward building a consistent evaluation design 
through further study of theory, watching the children for clues to 
their responses, and trying out promising ideas that emerged from study, 
observation, or discussion. The design can then be implemented the 
following year, when some of the units will be in the final testing stage, 

B. EVALUATION OF THE UNITS 

The procedure for evaluating the newly-developed units varied within a 
general format having the following sequence: 

1 . Writing th e units 



The design for the literature subprogram specifies a range of elements 
(groupings of the kinds of experiences literature deals with) to be 
covered by the curriculum. These elements are further divided into 
components, each of which is concerned with some aspect of the element. 
For example, the element Make Believe consists of the components 
Little People, Wishful Thinking, Magic and Wonder, and Fabulous Crea- 
tures. 

Usually a project planner selects or is assigned a single component 
for which he takes the primary responsibility, 'Jhe planner outlines 



This portion of the report was originally written by Anne Nicol and 
Norma Ostrander. 
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units which are grouped into "cont-^'ts" within the component, and by 
interacting continuously with other staff members who serve as consult- 
ants and critics, creates a ur.it which is then rough-tested in the 
classroom. 

2, Classroom testing and revisions 

There are six classrooms available for the rough-testing, three at the 
University Laboratory School and three at Kalilii-Uka School. Some 
lessons are evaluated in several classrooms, others in only one. The 
planner responsible for the unit observes the rough-testing. In addi- 
tion one or two other planners are usually present to keep notes and 
provide additional viewpoints for the evaluation. The observations 
are recorded in narrative form and copies are given to the unit planner 
and to the planner in charge of testing. Classroom teachers often 
make their own records of the evaluation and share their insights with 
the planner. 

There are no formal pro'^edures for revision; it usually takes the 
form of a continuing interchange between the unit planner and others 
(including the classroom teacher) who have observed the unit in use. 

In an effort to systematize classroom observations and provide more 
orderly feedback on the rough- testing for future reference, several 
forms were draft^^d and used experimentally. But these proved to be 
unwieldy for recording on-thc-spot observations, and since the most 
immediate needs of the unit planner were met adequately by narrative 
reports, the forms seemed unnecessary, at least at the rough-testing 
stage. 

There is a second source of feedback, dach unit includes a number of 
suggested activities, many of which yield student products of some 
type, The planners feel that they gain considerable insight into the 
children's grasp of the content through seeing the original writings, 
drawings, or dramas inspired by the literary selections. 

^ • fi^^al test ing of the component 

When the units have been revised to the satisfaction of the chief 
planner, the entire component is taught as a unit and evaluated once 
more. Again, the evaluation process is quite informal, but by this 
time certain measures have been built into the units to provide both 
teacher and planners with clues to the students' '^progress." 

C. DESCRIPTION OF THE TESTING CONTEXT 

I, Students 



The classrooms used for testing at the University Laboratory School 
are \*ngiaded, containing children who would normally be in grades two, 
three, and four. They span the entire ability range of the school. 

One class each of second, third, and fourth graders at Kalihi-Uka 
School also took part. The third and fourth graders are high-ability 
students but each class has a number of slow and average ones as well. 
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Additional second and third grade classes were used occasionally at 
both schools to test some materials. 

2, Teachers 

Cooperation from all the teachers was excellent. They talked freely 
with planners and observers and were eager to learn all they c^uld to 
help them teach the materials effectively. Most of them seemed to 
appreciate demonstrations by the staff. They liked the planner's 
willingness to put himself in the teacher’s place. The teachers were 
able to tell the planners about such matters as student behavior, 
classroom control, and characteristics of elementary children that 
teachers know best. 

The problems encountered this year are attiibutable almost entirely 
to the fact that the curriculum requires a teaching style that is new 
to even the most experienced teachers. Because the program has such 
enthusiastic support from teachers who have used the materials, prob- 
lems in teaching should diminisli steadily as teachers become more and 
more familiar with the objectives, 

SOME RESULTS OF THE ROUGH-TESTING 

Six complete components were tested this year, However, since most of the 
testing was of single segments, it is inappropriate to speak of results 
since the units are still undergoing review and revision. The kinds of 
factors vhat call for changes after classroom testing generally fall into 
three categories: 

1. Faulty targeting 

A particular unit turns out to be too easy or too difficult for the 
ability group to which it was directed. In this case, the unit is 
simply moved into another context destined for other types of children, 
or a note to the teacher is included so that she niay use the unit more 
effectively. 

2. Lack of appeal 

A literary selection or activity simply does not evoke interest. Some- 
times the unsuccessful part is dropped, ^k're often, attempts are made 
to see why a theoretically sound approach did not succeed in practice. 
Often the teacher provides the necessary information for revision. 

3. Inadequate teacher orientation 

The teachers' notes do not provide adequate or appropriate information 
for teaching t^ie mit according to the intent of the planner. A 
failure of this type is usually apparent to the observing planners who 
ean be informed, again by the teacher, as to weaknesses in the unit 
p ans. 
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The components Fabulous Creatures, Narrow Escapes, Magic and Wonder, 
Insights, Little People, and Wishful Thinking »^ere tested during the 
school year and are now considered to be in final form. A record has 
been kept of the observers' comments and some of the major revisions 
made in the components as they were developed. Student comments were also 
noted and a file of students* stories and drawings was created. Such 
materials will be used by the planners both as they develop other compo- 
nents and train teachers for using them. One of the stories inspired by 
Fabulous Creatures attached as Appendix 41 is a good example of what the 
planners consider evidence that a lesson has evoked a desirable response. 

It is impossible to categorize meaningfully the evaluation findings from 
this year’s rough-testing. Especially at this stage, evaluation-revision 
is a constant cycle, so that criticisms are valid only at the moment they 
are made. The teachers who used the materials are enthusiastic, ai.d the 
students appear to have enjoyed and appreciated the new diet of literature. 
Later in this report, a special "evaluation unit" is described which pro- 
vides some further insight into the success of the literature program. 

E. EVALUATION OF STUDENT RESPONSES 



Throughout the year, meetings were held to discuss approaches to evaluating 
the response of the students to the program. For the most part, the dis- 
cussions got caught up in the difficulties and dangers of attempting to 
measure aesthetic response by existing methods and often produced nothing 
substantive. Some of the ideas brought forth are noted, however, so that 
they may be revisited later. Others, which seemed to have some practical 
application, were carried further, and finally, an evaluation device which 
was actually used and analyzed is described. 



While it is undesirable to evaluate the student by rating his responses, 
it does seem feasible to classify responses which wereTorth coming to a 
particular work. Alan Purves^"C^ategories of Response" was suggested as a 
possible starting point. Hi^ categories are engagement, perception, inter- 
pretation, and evaluation. Another descriptive device which might be used 
in the same manner is KraChwohl's taxonomy.® There are some strong objections 
to using this scheme in its hierarchical form, at least insofar as it implies 
rati ng the response of the child. Also, it was felt that a very important 
element which must be accounted for in describing response to literature, 
the "aesthetic process,” is not treated adequately in Krathwohl’s work. The 
problem in using a system developed from these categories is essentially 
one of defining typical behaviors for each of the categories so that they 
may be readily observed and recorded. Even assuming that a workable system 
could be developed, converting it from a description of events into an eval- 
uative instrument poses knotty problems. 
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Anotier aspect of the literature program which must not be overlooked is 
its humanistic goal: essentially, that the student of literature gain 

incri^asing understanding of man^s place in the world. It is not intended 
that this kind of understanding be evaluated directly, but that it be held 
in awareness as measures are developed for the more specific and more easily 
defitied objectives of the literature units. 

The 'brainstorming** sessions produced a number of ideas which were acted 
upon to various degrees. In line vath the idea that an evaluation must be 
consistent with the notion of initial response and that it should carry on 
the work of a unit rather than follow it, some evaluation defices were 
built into the units themselves. These took the form of puzzles, games, 
and dramas very much like the activities suggested in each unit. These 
built-in evaluation devices, however, were designed to be used by every 
group, and each had a well-defined purpose so that the observing planners 
could gather information immediately on certain aspects of the unit. 

Another notion which developed out of the evaluation discussions was to 
make constructive use of the "gems" which emerge occasionally and unexpec- 
tedly during the presentation of the units. "Gems" are responses that 
show unusual and unanticipated insight into some aspect of the story. 

In addition to recording such responses, it would seem useful to share 
them with teachers through the instnjctional manual as examples of what 
could happen, but not as guides, directives, or standards. 

From the planners* notes to the teachers, the beginning of a list of 
question areas (which attempt to assess the children's understanding of 
significant relationships in the stories) was compiled along with reasons 
for asking such questions. As more components are completed the list may 
be expanded and improved upon, so that eventually it can serve as a source 
of items and ideas for evaluations of the sort that were piloted at the 
close of this year. 

An experimental test was developed and tried out in one of the classes 
that had used a number of the literature contexts during the year, The 
test consisted of (1) the presentation of a new and quite sophisticated 
story, (2) an evaluation to be written by each student and (3) an inter- 
view using several sorting games and a series of word associations. It 
was intended to be a learning experience for each child as weTl as an 
evaluation of his progress in the program, and it was designed to provide 
feedback to both teacher and planner about individuals as well as about 
the class as a whole. 

The structure of the test provided a profile for each student with scores 
in (1) understanding of the structure of the story, (2) understanding the 
work as the outcome of the author*s choices or intents, (3) understanding 
of or responding to the characters, and (4) responding to certain special 
qualities of the work. A variation in types of test items, from written 
to oral, highly-structured to open-ended, was provided both for experi- 
mental purposes and to evaluate fairly children whose responses are re- 
vealed in vaiy'ing modes. 
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The results were quite encouraging. The test proved to be valid for dis- 
criminating between the higher and lower-performing students in the class. 
Furthermore, it provided the teacher with some new insights into certain 
students who had been 1:00 shy to participate actively in classroom discussions. 
Some of the items did not have a single correct answer, yet procedures were 
developed so that these items could be included in the profiles for the stu- 
dents. Response to character was assessed using several semantic-differen- 
tial types of items, even though there had been some question about the chil- 
dren's ability to use this format. The responses to these suggest that the 
children were quite capable of responding to the scales provided. 

In addition to information about the students, ideas for revising the tests 
were collected in the process of administering and scoring the responses. 

The children themselves had some helpful comments. On the whole the test 
seemed to be valid enough so that some tentative conclusions could be drawn. 

The children had most difficulty in discerning the author’s intent, with 
reference to his reasons for building a character into the story, and to 
his purpose in telling the story at all. They also had some trouble grasping 
the structure of the entire story. Still, there were some surprisingly good 
responses to these questions, some of which were difficult even for the 
planners. The students tended to be more competent in describing and respond- 
ing to the characters in the story and in identifying their "'special quali- 
ties.” 

Although the test was based on a single story, the questions required that 
the child draw on literary experiences gained throughout the year. That 
is, most of the children simply would not have known how to approach many of 
the questions at the beginning of the year because the concepts had never 
before been presented to them. This hypothesis could be tested by using a 
similar instrument with new students at the beginning of the year. 

The two forms which evaluation of the literature program took during the 
year 1968-69 have been fruitful. The first, evaluation of the units them- 
selves, is really an indispensable part of the writing process. The high 
degree of cooperation among project and teaching staff made this process 
a smooth-flowing and productive one. The second foim, evaluation of student 
response, barely got off the ground. The progress made, however, seems to 
be in the right direction as far as the theory of the project is concerned. 

It also was strongly supported by Alan Purves who served as an outside eval- 
uation consu)tant. The recurring difficulties in working with responses have 
been noted, recorded, and dealt with in such a way that they will at least 
provide a co.utinuin^ and uplifting objective for evaluation activities to 
follow. 
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V. THE LANGUAGE SYSTEMS SUBPROGRAM^ 



A. DEVELOPMENT OF THE CONCEPT OF EVALUATION, 1968-1969 

The chief goals of the Language Systems subprogram are of three kinds: 
humanistic, disciplinary, and utilitarian. Because some of these goals 
tend to resist behavioristic reduction, there must be a continuing search 
for new definitions and strategies for evaluating the program. 

Humanistic goals concern learning something about one^s self from graduaUy 
accumulating (and occasionally heuristic) insights into the nature and use 
of language. (Language is defined as that system -- usually manifested 
in speech and writing ^hich makes possible to man the encoding, manipu- 
lating, constructing, and sharing of his public and private experience.) 
Progress toward such a goal is difficult to test; it would be reflected 
ultimately in an attitude toward language and the study of it which could 
be characterized as flexible, tolerant, questioning, respectful, humble, 
and open-minded. 

One of the two major disciplinary (that is, content) goals for students 
is to learn about linguistic concepts and the underlying principles that 
appear to govern language through study of some of their very specific 
manifestations in certain topics in language, communication, and culture 
selected for the program. Again, progress would be difficult to measure 
since students’ insights are expected to be largely cumulative. In the 
program students are given numerous opportunities to grasp the notions of 
language as system, as creative, as the substratum of social cohesiveness. 
Thus, if the insight is not achieved one time, it may well be another time. 
At the elementary level of the program it is expected that students will 
learn about the internal structure of language, about its uses, and about 
people's attitudes toward it, but all in a very informal and implicit way. 
It is in the intermediate and high school years that students will cover 
some of these same concepts and principles formally and explicitly. At 
that time it is expected that their preparation in the elementary program 
will stand then, in good stead. It is unlikely, however, that this poten- 
tial can be measured while the students are yet in the elemental^/ program. 

But the major disciplinary goal is the development of skills commonly 
used In the processes of scientific inquiry. Since these skills are moie 
easily identified, mastery of them is more readily assessed. Observing, 
classifying, measuring, inferring, using notational systems, interpreting 
data, reporting, experimenting, hypothesizing, and predicting are bf.sic 
processes in discovering and developing linguistic knowledge. Many of the 
activities which constitute the units require the use of one or 7tx>re of 
these process skills. Hence the accomplishment of a given activity is 
itself a measure of the extent to which the skill has been mastered. In 
the elementary units this mastery of skills is being indicated on a self- 
tracking personal progress report. 



This section of the report was written by Don Sanborn and RutJi Crymes. 



Lesson materials specify minimal levels of performance both in knowledge 
and in skills for each group of activities. Additional activities and 
worksheets provided for supplementary or optional use indicate perform- 
ance beyond the basic level. 

Beginning in the early stages of the project with attempts to specify 
behavioral objectives and to devise conventional achievement tests for 
them, the planners have come to rely more and more on pre and post 
measures, observation of the students at work on the activities and games, 
examination of students' products, and teachers' evaluations for assessing 
mastery of the major concepts and skills in a particular unit. Such 
judgments are admittedly subjective; nevertheless they yield some insights 
into the degree of self-direction the students are able to achieve in the 
unit. That is, given the data and suggested methods for handling the 
data, an attempt is made to answer the question: Are the students taking 

hold and proceeding toward some conclusions about language -- or about 
communication in general -- and are they enjoying what they are doing? 

The utilitarian goal of improving the language skills of students appears 
less difficult to assess. Yet there is scarcely any experimental evi- 
dence showing a correlation between knowledge about language and the 
ability to use language. Still, it seems plausible that, if students 
enjoy the study of language, their attitude toward language in general 
and toward the study of English in particular might become more positive, 
with some salutary (though indirect and unprovable) influence on their 
skills in the language. 

B. OVERVIEW OF LANGUAGE SYSTEMS UNITS 

Language Systems is a subprogram in the study of communication and language, 
designed to begin in grade four. Units are at various stages in the devel- 
opment-testing-revision cycle. They are listed below; the ones already 
tested are described briefly, 

1 . Elementary units already tested in University Laboratory School classrooms 

a. The Advertising Unit deals primarily with linguistic devices common- 
ly used in magazine, radio, and television ads: puns, alliteration, 

substitution of animate qualifiers for inanimate, and the like. 

The students' data are drawu primarily from a booklet of magazine 
ads, tapes of radio commercials, and films of television commer- 
cials. Students classify ads, compare language devices, and write 
their own ads. 

b. The Animal Communication Unit explores non-verbal communication, 
systems of animal messages, and the training of subjects (such as 
porpoises) to understand human communication. Data are supplied 
through tradebooks, tapes, and film loops; students prepare graphs, 
traveling games, and other kinds of reports showing features of 
animal messages and comparing them with human ones. 
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c. The International Language Unit introduces the students to 
Esperanto as an exaJiple of an artificial language. Students 
leam its sound and writing system, some of its vocabulary, 
some of its devices for making words, its syntax, and then pro- 
ceed to compare it v:ith English and to invent their own language. 
Data are provided primarily in the text and reference library, 
and students conduct such investigations as writing a simple 
grainmar for Esperanto and for their invented languages. 

d. In the Sign Language Unit students look at hula gestures, deaf 

sign language, and Indian sign language with a view to understanding 
how an unspoken and unwritten form of communication might work. 
Students teach each other to send messages in some of these forms 
and contrast them with human speech and writing. 

e. The Symbol Systems Unit introduces students to pictorial languagc:> 
^►:ch as international travel and highway symbols, map symbolism, 
and a system called Semantography which resembles Indian picture 
writing. The unit concludes with a brief look at syllabic and 
alphabetic writing systems. Students variously invent and test 
international symbols; investigate specialized visual, aural, 

and tactile symbol systems; and compile dictionaries based on their 
research. 

2 , Elementary units in component and simulation testing phases 

a. Dialects 

b. Social Formulas 

c. Sounds 

3. Elementary u n its in development 



a. 


Gestures 


b. 


Names 


c. 


Propaganda 


d. 


Secret Codes 


e. 


Writing Systems 


Elementary units projected 


a. 


Backgrounds of English 


b. 


Popular Songs 
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5. Intermediate School Units 



Three units for junior high school have been completed. Further 

development awaits completion of the elementary program. 

a. The Historical Development of Vocabulary Unit focuses on the 
history of English as seen through its borrowings and on its 
cultural contacts as reflected in such things as place names. 
Students attempt to arrive at conclusions about conditions 
necessary for the replacement of one language by another, the 
relative frequency and types of lexical items coming from various 
cultural contacts, and the effects of dictionaries and conscious 
attempts to enrich the language on English. 

b. The Phonology Unit deals with the nature and histor>* of English 
consonants, vowels, and the interrelations of sound and spelling 
on the modem English spelling system. Students conduct experi- 
ments tc distinguish between the perceptual features of conso- 
nants and vowels in English, investigate the rules that govern 
initial consonant clusters, reconstruct the great vowel shift, 
and consider the problems of spelling reform. 

c. The Words Unit treats parts of speech in English, how one part 
of speech is changed to another, the basic syntactic features 
of nouns, and verb tense. Students manipulate examples of parts 
of speech to arrive at generalizations about what constitutes 

a word class, investigate the processes of affixation and co- 
occurrence, and attempt to formalize the English tense system 
in terms of mathematical logic. 

C. EVALUATION OF THE UNITS 

Language Systems units currently in development are undergoing a far more 
rigorous cycle of testing even before they reach the classroom than 
heretofore was customary, Component and simulation testing now precede 
testing in schools. Component testing is the trial of selected lessons 
(or simply a handful of ^terTal^ wi^ a small group of students or even 
an individual to determine whether youngsters of the target p.ge-group 
can in fact perform the required tasks. Simulation testing is a tryout 
of the completed unit in first-draft form with a small group of teachers 
and planners who act both as students and as critics. Soth cycles of 
testing have been found extremely helpful in producing a more polished 
form of the unit for classroom tryouts in the University Laboratory School. 

Classroom trials have usually taken up fifty-minute periods for fifteen 
consecutive days. The first and third days each week were generally used 
for whole class meetings for presentation of the topic, problems to be 
investigated, and exemplary research studies. The remaining days were 
generally devoted to work individually, in pairs, or in small groups. The 
room was often divided into informal working and resource areas where 
students could browse in library materials, tapes, games, and film loops. 
Activities, worksheets, and independent research have been largely self- 
directing and self-checking. 
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Evaluation of the materials tested to date has taken the form of informal 
notes or reports submitted by outside observers, teachers, the planner 
and by students. The forrvs used for these are on file. It would be 
difficult to assess the effect of the customary practice of teacher-planner 
conferences on the success of the units tested on campus. The planners 
are now generally convinced that the information most useful in revising 
the units is that provided by the teacher, especially if the comments deal 
specifically and in detail with specific exercises and activities in rela- 
tion to her own teaching strategy and her particular students. Such infor- 
mation clearly defines where the unit is strong and where it is weak for a 
specific teacher with a particular level of students. Consequently, adjust- 
ments are being made to the teacher's manual to insure that this information 
reaches the planner regularly. 

The kinds of changes made in the units as a consequence of the limited 
testing typically include the following: activities have been re-ordered 

to place the more familiar and accessible material at the beginning; formats 
have been modified to make them more attractive; entire activities have 
been abandoned and others added; and the suggestions to the teacher have 
been enlarged on the basis of specific recommendations by the testing teacher. 
More specific changes (actual and proposed) include attempts to make activi- 
ties still more self-directing and self-checking than heretofore; to develop 
some relatively simple classiHcation exercises to provide practice for stu- 
dents who need it; to delineate cognitive, linguistic, and skill goals more 
clearly for the teachers; and to rely more and more on manipulative activi- 
ties, including card games and puzzles, for instruction as well as for review. 

Since testing thus far has involved a different group of students for each 
unit, achievement testing has not been possible. The micro-pilot program 
beginning in the fall of 1969 will provide, for the first time, groups of 
students who have taken four consecutive elementary Language Systems units. 

At that time it is expected that planners will be able to observe the cumula- 
tive effect of the units as well as the degree of success when taught to 
classes of high, average, and low abilities. It is also expected that the 
micro-pilot testing program will provide the project with a core of persons 
somewhat experienced in the use of the Language Systems materials who can be 
called upon to assist with teacher preparation as well as revisions of the 
materials for wider testing. 

The planners generally agree that the following are the most important conclu- 
sions to date: 

1. Children are interested in the language systems topics. 

2. ^ They do grasp quite complex language concepts when they are allowed 

to wo7f them through by themselves through trial and error. 

3. Students provided with the resources and open-ended situations built 
into the Language Systems unit, and encouraged to draft, test, and 
revise their own generalizations can learn about language systems. 

4. The teacher is a major factor in the success of the unit: her interest, 

skill, and cooperation are vital. 
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VI. EVALUATION OF THE 1968-69 EVALUATION 



The Hawaii English Project has developed a wide spectrum of programs 
demanding an evaluation effort flexible enough to run the gamut from units in 
language skills with highly specific behavioral objectives to those in litera- 
ture which tend to defy the behavioristic viewpoint in favor of a humanistic 
response to literature, In each case, evaluation activities have been designed 
to fit the philosophy of the programs, not to reshape programs in accord with 
any philosophy of evaluation. 

The primary purpose of this year's evaluation was to provide curriculum 
planners and administrators with detailed information useful in revising objec- 
tives, cux*riculum materials, and instructional procedures. In order to conserve 
the resources of this evaluation, a decision was made to focus major attention 
upon the Language Skills subprogram, since it was the one furthest along in field 
testing and the first to be selected for large-scale implementation in Hawaii. 

The comments of the planners and the revisions supported by evaluation evidence 
validate the success of the year's effort to provide planners with information 
essential to revising their materials. Several factors helped to account for 
this success. First, the planners worked with the evaluation specialist in 
devising specific questions to be answered with supporting evidence to guide them 
in revising their programs, A second factor was the direct involvement of the 
planners themselves in observing and working with children, and in talking with 
teachers and other personnel. Third, the evaluation specialist attended the 
weekly staff meeting of the planners, listened to their plans and problems, and 
gave them direct feedback and interpretation of evaluation information as soon 
as it was available. 

This past year's evaluation is considered by the evaluator to have been less 
than satisfactory in assembling information useful to the project manager and 
che administrators of the Hawaii Curriculum Center. There are several reasons. 
First, the infoimation needs of the curriculum planners and section chiefs took 
priority, leaving less time for working directly with the project manager and 
the director of the Center in determining what decisions they would need to make 
during the year and what sorts of evaluation evidence they \ ould need. Second, 
some of their information needs had more to do with project management than with 
curriculum revision; to meet those needs would have required a shift in the 
direction of the evaluation. Ways to improve this aspect of the evaluation task 
will be explored in the coming year. 

Feedback on specific evaluation instruments used during the year has been 
useful in directing needed revisions in their construction and administration 
procedures. For example, most of the instruments administered once during the 
year (such as the student media preference inventory or the teachers' rating 
scale for materials) were of limited usefulness since they show'ed ratings at 
only one point in time. Next year similar instruments will be administered to 
random samples of students and teachers throughout the year in order to monitor 
their changing attitudes as certain materials come into wider use while others 
are completed by most students. 
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A few evaluation procedures tried this year (case studies, for example), 
while appearing to be conceptually appropriate, fell short of their potential 
because of failure to insure prompt and frequent contacts between the persons 
gathering the data and the planners, teachers, and administrators who might 
have gained from the insights suggested by the data. Periodic meetings of 
this nature will be held next year, and a more systematic method of providing 
feedback to planners, administrators, and teachers will be initiated. 

Exploratory efforts to evaluate components of the English curriculum less 
amenable to behavioral definition have provoked much interest on the part of 
both evaluators pd curriculum planners. Perhaps the greatest contiibution 
that the evaluation attempts in this area have made is the indirect benefit, 
shared in by all, of a clearer understanding of how students are responding 
to literatuie. 
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